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This  paper  argues  that  the  prima  facie  evidence  suggests  that  the 
World  Bank  is  not  particularly  effective  either  in  dealing  with 
countries  that  default  or  in  promoting  countries,  projects  and  ideas 
that  are  likely  to  do  well.  We  argue  that  this  is  probably  related  to 
the  fact  that  the  Bank  does  not  make  adequate  use  of  scientific 
evidence  in  its  decision-making  and  suggest  ways  to  improve 
matters. 
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I.  What  can  the  World  Bank  do? 

Even  in  today's  world  of  burgeoning  private  capital  markets,  there  is  much  that  an 
organization  like  the  World  Bank  can  contribute  towards  realizing  the  dream  of  a  world 
without  poverty.  First,  of  course,  there  is  emergency  relief:  At  any  point  of  time  there 
are  people  in  the  world  who  are  in  the  middle  of  an  acute  crisis  and  need  emergency 
relief  While  delivering  relief  effectively  is  not  easy,  there  is  broad  agreement  that  it  is  a 
worthwhile  goal.  However  the  World  Bank  believes,  rightly,  that  this  is  not  enough.  As 
we  see  it,  there  are  at  least  three  important  ways  it  can  contribute  to  the  broader  project: 

1.  Default  Management:  Optimal  default  management  suggests  that  countries  that  have 
defaulted  should  be  rationed  in  their  access  to  credit,  without  being  starved  of  the 
liquidity  needed  for  successful  restructuring.  Individual  lenders  might  either  be  too 
willing  to  lend  compared  to  this  optimal  plan  (because,  say,  post  default,  the  country's 
balance  sheet  looks  much  better)  or  too  reluctant  (because,  an  efficient  restructuring 
benefits  all  potential  lenders).  An  important  potential  role  for  multilateral  lending 
institutions  is  to  help  the  lenders  coordinate  and  thereby  maintain  the  right  balance  of 
incentives  and  liquidity.  In  practice,  while  the  IMF  handles  the  default  itself,  the  eventual 
restructuring  package  typically  includes  the  Bank  and  its  sister  organizations. 

2.  Promoting  economic  opportunities:  In  developing  countries,  economic  opportunities 
often  need  to  be  created  and/or  promoted  before  the  market  can  find  its  way  to  them.  First 
because  there  are  physical  externalities  between  investments:  The  benefits  from  some 
investments  (infrastructure,  courts,  etc.)  are  only  realized  after  other  complementary 
investments.^  Second,  there  are  informational  externalities:  New  countries  and  new 


^  As  in  Murphy,  Shleifer  and  Vishny  (1989). 


projects  are  associated  with  uncertainty  that  gets  partially  resolved  after  the  initial 
investments.  The  early  investors  are  therefore  doing  other  investors  a  favor  and 
consequently,  the  market  under-supplies  such  investment.  Because  the  World  Bank  can 
afford  to  lose  some  money,  it  can  internalize  this  externality.  Moreover  its  country 
expertise  might  allow  it  to  select  better  initial  projects  and  thereby  minimize  risk. 
3.  Promoting  Best  Practice:  It  is  not  easy  to  be  aware  of  the  recent  history  of  the  world 
and  hold  that  countries  mostly  pursue  policies  that  best  serve  the  majority.  This  is  in  part 
the  way  of  political  economy,  but  ignorance  and  ideology  have  also  done  their  bit.  The 
World  Bank  can  do  something  about  this:  ^  First,  because  it  can  get  the  necessary  know- 
how.  Second,  because  it  can  get  country  governments  to  listen  to  it,  especially  if  it  has 
subsidized  loans  to  offer.  And  third,  because  country  governments  can  use  World  Bank 
pressure  as  an  excuse  when  dealing  with  the  domestic  supporters  of  the  status  quo. 

II.  How  effective  is  the  World  Bank? 

How  well  does  the  Bank  do  along  these  three  important  dimensions?  There  is  a 
long  and  largely  unresolved  debate  on  whether  multilateral  lending  (or  aid)  has  any  effect 
on  overall  growth,  where  assigning  causality  has  been  a  persistent  problem.  We  are 
interested  in  the  narrower  question  of  identifying  specific  charmels  of  influence,  where 
the  causality  issues  are  exaggerated  by  the  lack  of  data.  Our  regressions  results  are 
therefore  best  seen  as  an  attempt  to  check  whether,  prima  facie,  the  conditional 
correlations  in  the  data  are  consistent  with  the  models  we  have  suggested. 

Optimal  default  management  implies  that  overall  credit  access  for  the  country 
should  go  down  after  a  default,  to  preserve  the  incentives.  However  multilateral  lending 
might  go  down  after  default  but  it  might  also  go  up,  if  the  private  market  overreacts. 


To  see  how  default  actually  gets  managed,  we  use  the  BIS-IMF-OECD- World 
Bank  statistics  on  external  debt/*  We  define  multilateral  lending  to  include  all  loans  from 
all  continental  Development  Banks,  the  World  Bank  and  the  use  of  IMF  credit.^  Default 
is  measured  by  the  country  having  signed  an  agreement  with  Paris  Club  creditors.^ 

.  We  regress  Log[net  stock  of  multilateral  lending  to  a  country  in  2002]  on  Log[net 
stock  of  multilateral  lending  to  a  country  in  1996],  a  dummy  for  having  defaulted 
between  1990  and  1995,  another  for  having  defaulted  between  1996  and  2002  and  the 
interaction  of  the  two.  The  results  are  shown  in  column  1  of  Table  1.  Both  current  and 
past  default  is  positively  correlated  with  World  Bank  lending  but  defaulting  twice  has  a 
negative  sign.  When  we  run  the  same  regression  for  market  credit  we  find  a  very  different 
pattern:  current  default  has  a  negative  correlation  with  lending  while  past  default  is  not 
significant  (column  2).  As  a  result  of  the  two  countervailing  effects,  there  is  no 
correlation  between  total  credit  and  current  default,  and  a  positive  correlation  with  past 
default  (column  3).  Conditioning  on  a  number  of  measures  of  the  state  of  the  economy  in 
1996  such  as  growth  rates,  inflation  rates  and  change  in  openness  over  the  1990-96 
period  (from  the  Perm  Tables,  Heston  et  al.,  2002),  leaves  the  pattern  of  correlations 
unchanged  (columns  4-6).^  The  results  are  robust  to  the  inclusion  of  the  same  controls  in 
2002  and  a  control  for  the  budget  deficit. 


'  This  is  the  model  of  the  Bank  that  Gilbert,  Powell  and  Vines  (2000)  call  the  Knowledge  Bank. 

"*  See  hUp://vvwvv  1  .oecd.org/dac/debt/. 

^  Here  we  do  not  include  the  outstanding  balance  of  Brady  bonds  in  multilateral  lending,  but  the  results  are 

the  same  if  we  do. 

*  From  http://www.clubdeparis.org/en/index.php. 

This,  for  example,  makes  it  less  likely  that  the  default  was  already  anticipated  and  captured  by  the  1996 
loan  amount. 


Unfortunately  at  this  point  we  do  not  have  data  to  say  \\iiat  happens  to  interest 
rates,  though  clearly  they  do  not  go  up  enough  to  discourage  extra  borrowing.  With  this 
caveat,  we  can  say  that  we  found  no  evidence  for  effectiveness  in  default  management. 

If  the  World  Bank  is  interested  in  promoting  economic  opportunities,  its  goal 
should  be  to  target  its  loans  towards  countries  and  projects  that  are  better  than  one  would 
expect  based  on  publicly  available  data.  If  it  is  interested  in  promoting  best  practice,  it 
should  give  preference  to  countries  and  projects  that  best  approximate  best  practice.  Both 
the  first,  selection  argument  and  the  second,  incentive  argument,  have  the  implication  that 
multilateral  lending  should  be  positively  correlated  with  country  and  project 
performance,  controlling  for  observables  such  as  default  and  the  state  of  the  economy. 

To  check  this  we  first  add  to  the  loan  regressions  already  discussed,  the  Bank's 
evaluation  of  the  quality  of  governance  in  different  countries  in  2001  and  1997.^  The 
recent  Bank  publication,  A  Case  for  Aid:  Building  a  Consensus  for  Development 
Assistance  explicitly  claims  that  the  Bank  is  now  targeting  countries  with  good 
governance.  The  results,  reported  in  column  1  of  Table  2,  show  that  neither  of  the 
coefficients  on  the  governance  variables  are  even  remotely  significant,  and  one  is  actually 
negative.  An  alternative  possibility  is  that  the  Bank  is  targeting  excellence  in  project 
performance  rather  than  governance.  In  Column  3,  we  add  to  this  equation  an  average 
country  measure  constructed  from  the  Bank's  own  ratings  of  projects  funded  in  the 
country.    We  include  both  the  1990-95  average  and  the  1999-2001  average.     None  of 
the  four  performance  measures  are  significant  and  three  out  of  four  are  negative.  A  final 


Available  from  http://\\"\v\v. worldbank.org/wbi/govenance/govdata. html.  They  provide  six  indicators 
which  we  sum. 
From  the  Operations  Evaluation  Department  Annual  Reports 


test  along  these  lines  is  to  estimate  the  relation  between  World  Bank  lending  and  project 
performance  for  all  the  projects  evaluated  by  the  Operations  Evaluation  Department  at 
the  Bank.  For  these  we  estimate  a  relation  between  the  amount  committed  by  the  Bank 
and  its  outcome  rating,  controlling  for  the  sector,  the  country,  the  length  of  the  project, 
the  year  when  it  was  approved  and  the  closing.  The  coefficient  on  the  ratmg  is  negative  (- 
5.5)  but  not  significant  (the  t-statistic  is  -1.54).  However  when  we  replace  the  amount 
committed  by  the  fraction  of  World  Bank  funds  in  total  project  cost,  the  coefficient  is 
negative  (-0.28)  and  highly  significant  (t-statistic  of -3.27). 

The  results  based  on  the  project  score  have  the  problem  that  we  only  observe 
projects  that  were  chosen,  and  therefore  companson  between  them  understates  the  degree 
of  selectivity  exercised  by  the  Bank.  In  other  words,  the  Bank  could  still  be  discarding 
many  very  bad  projects.  Moreover,  our  analysis  does  not  distinguish  between  IDA  and 
IBRD  lending  and  it  is  possible  that  IDA  lending  is  more  performance-sensitive  than 
IBRD  lending.  To  the  extent  that  this  adds  measurement  error  to  the  independent 
variable,  it  will  tend  to  make  our  results  less  significant.  Note  however  that  many  of  the 
coefficients  are  actually  negative. 

In  A  Case  for  Aid,  the  research  staff  in  the  Development  Economics  Vice 
Presidency  of  the  Bank  make  a  series  of  claims,  based  on  cross-country  evidence,  about 
the  high  level  of  effectiveness  of  Official  Development  Assistance  generally  and  World 
Bank  lending  in  particular.  While  this  is  not  the  place  to  try  to  resolve  the  conflict 
between  their  claim  and  ours,  note  that  our  results  only  show  that  World  Bank  loans  do 
not  outperform  what  one  would  have  predicted  on  the  basis  of  the  state  of  the  economy. 


We  have  not  yet  managed  to  get  hold  of  the  1 996-98  numbers. 


Success  in  promoting  economic  opportunities  comes  from  getting  the  private 
sector  to  respond  to  the  Bank's  promotion  efforts.  One  test  of  this  is  to  see  if  private 
sector  lending  to  the  country  responds  to  the  Bank's  evaluation  of  governance  in  the 
country,  controlling  for  public  mformation  (default  history,  the  state  of  the  economy). 
This  is  a  very  weak  requirement  since  we  cannot  be  sure  that  the  measure  of  governance  • 
that  they  are  actually  using  comes  from  the  Bank  (commercial  firms  also  provide 
measures).  Column  2  in  Table  2  reports  that  the  governance  measures  are  essentially 
uncorrelated  with  private  lending.  A  more  demanding  requirement  is  to  see  if  private 
sector  lending  responds  to  results  of  Bank  projects  (i.e.,  the  average  project  score  used 
above).  Since  we  do  not  know  how  quickly  the  private  sector  responds  to  the  information 
we  include  both  the  1990-95  average  and  the  1999-2001  average.  The  results,  shown  in 
column  4,  confirm  the  lack  of  correlation  (one  is  positive  and  the  other  is  negative,  and 
the  t-statistics  are  less  than  1). 

This  evidence  certainly  does  not  rule  out  the  possibility  that  the  pnvate  sector 
does  respond  to  a  small  fraction  of  highly  successful  bank  projects.  But  it  does  question 
our  confidence  in  the  Bank's  ability  to  influence  the  private  sector. 

The  final  piece  of  evidence  is  on  the  Bank's  efforts  to  directly  influence  best 
practice  through  the  advocacy  of  certain  ideas  in  the  annual  World  Development  Report 
(WDR).  We  looked  at  the  number  of  citations  to  the  WDR  and  to  certain  key  phrases  in 
the  WDR  both  in  the  academic  press  and  on  the  web.  The  citations  fall  off  very  sharply 
after  about  the  first  two  years  since  publication. 


The  WDR  therefore  does  influence  the  pubhc  discourse,  but  only  temporanly.  It 
is  possible  that  this  is  in  the  nature  of  the  public  discourse,  but  it  does  raise  important 
concerns  about  the  power  of  ideas  in  this  context. 

III.  The  World  Bank  of  the  Future:  Towards  Greater  Effectiveness 

While  our  results  are  not  always  easy  to  interpret,  it  is  hard  to  point  to  a  single 
piece  of  evidence  that  clearly  signals  effectiveness.  With  respect  to  the  management  of 
default,  it  does  not  look  that  there  is  effective  coordination  among  the  lenders.  In  part  this 
might  reflect  the  political  compulsions  to  bail  out  private  lenders.  In  part  it  may  reflect 
genuine  disagreements  between  the  lenders  (the  Bank  and  the  Fund,  for  example)  about 
how  much  punishment  is  optimal.  It  remains  however  that  incentives  for  potential 
defaulters  are  probably  not  what  they  should  be  and  in  the  end,  this  hurts  the  borrowers. 

Establishing  a  clear  set  of  rules  for  dealing  with  default  and  enforcing  them 
(including  by  punishing  deviant  lenders)  is  therefore  vital.  This  is  the  natural  preserve  of 
the  IMF:  The  Bank  should  accept  the  IMF's  lead,  once  they  have  agreed  on  the  rules. 

In  trying  to  understand  the  Bank's  limitations  in  promoting  economic 
opportunities  and  advocating  best  practice,  we  studied  in  some  detail  a  recent  Bank 
publication— Empowerment  and  Poverty  Reduction:  A  Sourcebook.  This  is  meant  to 
provide  a  catalogue  of  what  the  Bank  believes  are  the  right  strategies  for  poverty 
reduction.  A  subset  of  strategies  recommended  include:  computer  kiosks  in  villages;  cell 
phones  for  rent  in  rural  areas;  scholarships  targeted  towards  girls  who  go  secondary 
school;  schooling  voucher  programs  for  poor  children;  joint  forest  management 
programs;  water  users  groups;  citizen  report  cards  for  public  services;  participatory 
poverty  assessments;  internet  access  for  tiny  firms;  land  titling;  micro-credit  based  on 


group  lending;  and  many  others.  While  many  of  these  are  surely  good  ideas,  the  book 
does  not  tell  us  which  of  these  have  been  subject  to  a  randomized  evaluation  or  even  a 
non-randomized  evaluation  based  on  a  convincing  quasi-experimental  design.  In  fact,  to 
the  best  of  our  knowledge,  only  one  of  these  strategies  has  been  so  evaluated  in  a  LDC 
context — schooling  vouchers  for  poor  students — and  that  was  because  the  Colombian 
voucher  program  had  built-in  random  allocation."  Yet  it  receives  no  more  weight  than 
any  of  the  other  programs.  Moreover  for  many  of  the  programs,  the  book  makes  no 
mention  of  rates  of  return,  so  it  is  not  clear  how  one  is  to  choose  between  them.  Where 
the  rates  of  return  are  mentioned,  we  are  not  told  how  these  are  calculated,  and  in 
particular,  how  they  solve  usual  identification  problems.'   And  some  of  the  programs, 
such  as  the  Gyandoot  program  in  Madhya  Pradesh  in  hidia  which  provides  computer 
kiosks  in  rural  areas,  are  declared  to  be  successes  even  while  the  text  acknowledges  that 
the  computers  often  do  not  work  and  that  they  are  all  losing  money  (page  109).'^ 

Given  this,  one  should  not  be  too  surprised  if  the  Bank  is  not  very  successful  in 
identifying  good  economic  opportunities  or  if  the  private  sector  failed  to  follow  its  lead. 
And  policy  makers  may  be  forgiven  if  they  resist  the  Bank's  views  on  best  practice. 

If  the  Bank  has  to  be  more  effective  in  its  two  promotion  roles,  it  has  to  move  to  a 
much  more  scientific  process  for  the  selection  of  projects  and  ideas.  If  something  has  to 
get  the  Bank's  imprimatur,  the  expectation  should  be  that  there  is  a  clear  and  well- 
understood  theory  behind  it,  and  that  it  has  been  subjected  to  multiple  randomized 
evaluations,  to  ensure  both  internal  and  extemal  validity.  If  governments  are  reluctant  to 


"  See  Angristetal  (2001). 

'^  A  Case  for  Aid,  takes  a  similar  cavalier  approach  to  project  appraisal. 

'^  It  is  not  that  successful  projects  cannot  lose  money:  But  when  it  is  said  that  they  are  losing  money 
because  the  computers  often  do  not  work  and  there  is  not  enough  demand,  one  must  wonder. 


randomize,  the  Bank  can  collaborate  with  NGOs.  Success  in  these  evaluations  does  not 
need  to  be  judged  by  a  uniform  standard  across  countries — even  a  little  bit  of  success 
may  be  valuable  where  nothing  works.  But  the  standard  should  be  explicit  and 
determined  ex  ante.  The  process  of  trial  and  error  should  be  made  relatively  public,  as  a 
way  of  resisting  political  interference.  Finally,  where  randomization  is  physically 
impossible  (say  the  bank  needs  to  decide  whether  to  push  for  the  abolition  of  power 
subsidies),  the  rule  should  be  to  stay  away  unless  the  theoretical  case  and  the  available 
high  quality  non-experimental  evidence  is  exceptionally  compelling. 

This  could  mean  that  for  some  time  the  Bank  will  have  more  money  than  it  needs 
to  finance  specific  initiatives— though  going  to  global  scale  on  all  the  programs  where 
there  already  is  strong  experimental  support  (de-worming,  vitamin  A  supplements,  etc.) 
need  not  be  cheap.  There  is  however  always  relief  work,  which  might  include  broad 
budgetary  support  to  bankrupt  governments  (subject  to  what  we  say  above  about  default 
management) — if  only  to  protect  the  programs  for  the  poor  from  being  cut.  It  may  be 
justifiable  to  use  the  resulting  leverage  to  eliminate  policy  distortions  in  the  country,  but 
only  if  the  case  for  intervention  is  exceptionally  strong.  And  even  in  such  cases,  a  clear 
separation  between  with  budgetary  support  negotiations  and  the  recommendations  of  the 
Bank  for  long-term  development,  must  be  institutionalized. 

Effective  promotion  may  also  require  the  Bank  to  signal  its  commitment  to  an 
idea  by  putting  its  own  money  at  risk  of  default.  In  this  sense,  moving  completely  to  grant 
financing,  as  recommended  by  the  International  Financial  Advisory  Commission 
(Meltzer  Commission)  report,  may  be  counter-productive.  On  the  other  hand,  since 
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NGOs  may  be  more  willing  to  try  out  new  ideas  than  governments,  the  Bank  should  have 
a  grant  fund  that  can  be  used  to  finance  these  efforts. 

The  Bank  is  attracting  a  lot  of  unwelcome  attention  these  days.  A  Bank  that 
resists  advocating  what  it  cannot  prove  will  soon  stop  being  the  prime  target  of  the 
Seattle  crowd.  It  would  live  quietly  in  the  shadows,  doing  its  best,  doing  good. 
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Table  1:  OLS  Regressions  of  Log  of  Stock  of  loans  in  2002 


Regular  OLS 

+  Past  Controls 

Variable 

World 
Bank 

Market 

Total 

World 
Bank 

Market 

Total 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

log(LOANS) 

0.8776* 

0.7804* 

0.8061* 

0.9091* 

0.7717* 

0.8472* 

(1996) 

[0.05] 

[0.03] 

[0.04] 

[0.04] 

[0.03] 

[0.02] 

DEFAULT? 

0.4765* 

-0.6879* 

-0.0845 

0.6669* 

-0.3263 

0.1300 

(1996-2002) 

[0.1] 

[0.2] 

[0.1] 

[0.2] 

[0.2] 

[0.2] 

DEFAULT? 

0.4709* 

0.1905 

0.2238* 

0.4876* 

0.3318* 

0.2587* 

(1990-1995) 

[0.2] 

[0.2] 

[0.1] 

[0.2] 

[0.1] 

[0.1] 

INTERACT 

-0.6980* 

0.0596 

-0.3097* 

-0.7627* 

-0.3334 

-0.4412* 

[0.2] 

[0.3] 

[0.2] 

[0.2] 

[0.3] 

[0.2] 

Past  Controls? 

No 

No 

No 

Yes 

Yes 

Yes 

R' 

0.86 

0.89 

0.89 

0.89 

0.90 

0.93 

n 

144 

163 

170 

103 

107 

108 

Notes  -  Robust  standard  errors  are  in  parentheses. 

*  Significantly 

different  from  0  at  the  10-percent  level. 
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